feat(read-state): multi-slot splitting + no-op suppression for oversized blobs#1309
Merged
Merged
Conversation
wpfleger96
force-pushed
the
duncan/readstate-semantic-eviction-multiSlot
branch
from
6b33140 to
e4b48e7
Compare
…zed blobs When the read-state blob exceeds NIP-44's plaintext cap, split channel keys round-robin across up to 8 kind:30078 slots. Thread/msg entries stay in the primary slot and are trimmed to budget. No-op suppression skips redundant publishes when the blob hasn't changed. NIP-09 delete events clean up stale extra-slot blobs when transitioning back to single-slot mode. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96
force-pushed
the
duncan/readstate-semantic-eviction-multiSlot
branch
from
e4b48e7 to
081f3f4
Compare
wpfleger96
changed the title
…-ID persistence guard The splitContextsIntoSlots JSDoc still referenced 'already semantically evicted' after semantic eviction was removed. Updated to match the current behavior. Added a comment explaining why length-only comparison is sufficient for the extra-slot-ID persistence check. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
ReadStateManagerpublishes a single encrypted JSON blob (kind:30078) containing all read-state entries. As users accumulate channels and thread/message entries, the blob exceeds NIP-44's 65,535-byte plaintext cap — causing"publish failed: events too long"and breaking cross-device read-state sync.Solution
Three complementary mechanisms ensure the blob always fits:
Multi-slot splitting
When channel keys alone exceed the 32 KB self-imposed budget (even after evicting all thread/msg entries), partition channel keys round-robin across up to 8 separate kind:30078 events, each with its own
read-state:<slotId>d-tag. Thread/msg entries go into the primary slot only and are trimmed to budget there. Extra slot IDs persist tolocalStorageso they survive restarts. On fetch, all own-slot blobs are max-merged per key.No-op suppression
Before publishing, compare the about-to-publish contexts against
lastPublishedContexts. If identical, skip the publish entirely — avoids wasting bandwidth and relay writes on every 5-second debounce cycle when nothing changed.NIP-09 cleanup
When transitioning from split mode (multiple slots) back to single mode, publish kind:5 delete events for stale extra-slot blobs. Without this,
fetchOwnBlobBeforePublishkeeps finding old blobs, merging stale keys, and the no-op check never fires — causing an infinite re-publish loop.Files changed
desktop/src/features/channels/readState/readStateManager.ts—splitContextsIntoBudgetedSlots,publishSplitSlots,publishOneSlot,deleteExtraSlots, no-op suppression inpublish()desktop/src/features/channels/readState/readStateManager.test.mjs— 5 split tests + no-op suppression integration testdesktop/src/features/channels/readState/readStateFormat.ts—MSG_PREFIX,THREAD_PREFIXconstants (shared with byte-budget trim)desktop/scripts/check-file-sizes.mjs— updated override forreadStateManager.ts(1019 lines, queued to split)Testing
1242/1242 desktop tests pass. Biome clean. File-size check passes.